Mapping Eye Movements to Cognitive Processes

Eye movements provide a rich and informative window into a personÕs thoughts and intentions. In recent years researchers have increasingly employed eye movements to study cognition in psychological experiments, to understand behavior in user interfaces, and even to control computers through eye-based input devices. Unfortunately, like speech and handwriting, eye movements generate vast amounts of data with significant individual variability and equipment noise. Thus, the analysis of eye-movement dataÑthat is, determining what people are thinking based on where they are lookingÑcan be extremely tedious and time-consuming. Typical eyemovement data sets are simply too large and complex to be analyzed by hand or by naive automated methods. This thesis formalizes a new class of algorithms that provide fast and robust automated analysis of eye-movement data. Specifically, the thesis describes three novel algorithms for tracing eye movementsÑmapping eye-movement protocols to the sequential predictions of a cognitive process model. Two algorithms, fixation tracing and point tracing, employ hidden Markov models to determine the best probabilistic interpretation of the data given the model. The third algorithm, target tracing, extends an existing tracing algorithm based on sequence matching to eye movements. The thesis also formalizes several algorithms for identifying fixations in raw eyemovement protocols and provides a working system, EyeTracer, that embodies the proposed tracing and fixation-identification algorithms. To demonstrate the power of the proposed algorithms, the thesis applies them in three real-world domains: equation solving, reading, and eye typing. The equationsolving studies show how the algorithms can code, or interpret, eye-movement protocols as accurately as expert human coders in significantly less time. The studies also illustrate how the algorithms facilitate the prototyping and refinement of cognitive models. The reading study demonstrates how the algorithms help to evaluate and compare two existing computational models of reading and clear up temporal aspects of reading data using sequential aspects of the data. The eye-typing study shows how the algorithms can interpret eye movements in real time and help eliminate usability restrictions imposed by existing eye-based interfaces.